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Objective 

This  research  addresses  the  severe  power  constraints  and  bandwidth  limitations  in  the  observation 
of  the  battlefield  by  war  fighters  using  video  data.  New  dimensions  of  network  centric  electronic 
warfare,  such  as  the  Expeditionary  Sensor  Grid  will  further  stress  these  constraints.  We  develop 
an  algorithm  for  the  real-time  processing  of  mobile  video  sensor  data  by  devising  knowledge- 
based  methods  for  efficient  compression  and  coding.  Advances  in  data  compression  are  needed  to 
address  this  challenge.  Bandwidth  must  be  rationed  so  operationally  important  information  is 
transmitted  with  higher  resolution  than  extraneous  details. 

Approach 

We  formulate  mathematical  techniques  for  local  processing  of  raw  sensor  data  into  semantic 
information  with  flexible  resolution,  related  to  battlefield  objects  and  their  behaviors.  Three 
increasing  sequential  levels  of  compression  and  coding  are  developed.  They  are  being  tested  in 
laboratory  experiments  using  mobile  robots  and  implemented  in  a  prototype  system. 


The  first  level  of  compression  involves  a  functional  decomposition  of  video  data  into  spatio- 
temporal  wavelets ,  facilitating  the  semantic  interpretation  introduced  at  the  next  level.  The 
second  level  of  compression  groups  the  wavelets  into  sets  corresponding  to  moving  objects  and 
background  then  transforms  each  set  of  numerical  data  into  a  formal  language  and  transmits  only 
the  formal  grammars.  The  third  level  of  compression  creates  a  codebook  of  formal  grammars 
corresponding  to  generic  scene  components.  Formal  languages  created  from  sensor  data  will  be 
matched  against  codebook  languages  using  a  formal  language  measure.  If  a  match  is  found,  only 
the  codebook  index  need  be  transmitted. 
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Progress 


1.  Video  Processing  Testbed 

We  have  further  developed  a  video  processing  testbed  for  the  development  and  testing  of  our 
video  compression  approaches.  This  testbed  includes  implementations  of  all  algorithms  and 
software  mentioned  below. 

2.  Texture  Segmentation  Using  s-Machines 

We  have  developed  an  algorithm  for  e-machine  based  texture  segmentation,  which  is  more 
efficient  than  the  current  state  of  the  art  using  graph-based  methods.  This  segmentation  algorithm 
combined  with  the  development  of  Dynamic  Window  CCSA  (see  below)  allows  for  real-time 
segmentation  of  video  textures,  a  key  element  in  the  construction  of  our  code  book  testbed.  In 
addition,  to  improve  texture  segmentation  results  we  developed  and  tested  a  method  for  wavelet 
encoding.  This  encoding  extracts  relevant  dynamics  around  pixels  and  provides  these  as  easily 
symbolized  input  to  the  texture  segmentation  algorithm. 


Original  Image  Texture  Segmentation 


Figure  1.  Texture  Segmentation  Results  on  Robot  Image 
3.  s-Machine  Software 

Two  years  ago  we  designed  the  Clustered  Causal  State  Algorithm  (CCSA),  which  produces 
causal  state  machines  with  nearly  the  accuracy  of  the  Causal-State  Splitting  Reconstruction 
(CSSR)  the  current  state  of  the  art,  while  operating  ten  to  a  hundred  times  faster.  Last  year  we 
followed  this  up  by  developing  two  additional  refinements  of  CCSA. 

First,  we  implemented  a  version  of  CCSA  incorporating  the  k-means  approach  to  clustering, 
enabling  us  to  generated  machines  rapidly,  but  with  a  fixed  number  of  states.  The  accuracy  of  this 
algorithm  is  comparable  with  the  conventional  CCSA. 

Second,  we  implemented  Dynamic-Window  CCSA  and  algorithm  designed  to  fulfill  the  need  of 
texture  segmentation  (see  below),  which  requires  rapid  generation  of  machines  for  many 
consecutive  windows  of  the  symbol  stream,  by  use  of  a  sliding  window  of  information.  Combined 


with  statistical  metrics  DW-CCSA  automatically  group  patterns  together  in  a  symbol  stream, 
enabling  hierarchical  analysis  and  uncalibrated  phase-change  detection. 

4.  Mission-Oriented  Image  Metrics 

We  have  investigated  the  development  of  mission  oriented  image  metrics  to  better  compare  the 
utility  of  images  and  video  for  use  in  the  field.  We  have  developed  two  metrics,  both  using  a 
graph-based  region  segmentation  algorithm,  one  weighing  pixel-by-pixel  comparison  towards 
region  boundaries,  and  the  other  weighing  all  regions  equally  to  ensure  that  the  retention  of 
smaller  foreground  elements  was  not  outweighed  by  larger  background  features. 

5.  Code  Book  Testbed 

One  of  the  culminating  goals  of  this  project  is  the  development  of  a  code  book  testbed  to 
demonstrate  the  capabilities  of  converting  image  portions  into  code  book  elements  in  reducing 
bandwidth  costs.  The  basis  of  this  testbed  has  been  designed,  including  use  of  sockets  to  ensure 
realistic  bandwidth  measurement  and  incorporating  techniques  for  the  transmission  compression 
of  machines,  image  fragments,  and  segment  boundaries.  The  receiving  side  of  this  testbed 
incorporates  existing  texture  reconstruction  technology  to  be  run  in  real-time,  recreating  images 
based  on  code  book  entries,  which  are  determined  by  texture  segmentation  performed  by  DW- 
CCSA. 


Figure  2  -  Code  Book  System  Diagram 


6.  Conclusions 

We  have  assembled  and  vetted  the  components  of  a  semantic  video  image  compression  and  code 
booking  system.  Preliminary  integration  has  shown  the  compatibility  of  the  component 
technologies  and  suggests  future  applications  for  both  the  system  and  its  components. 
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